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A Comparison of Predictors For First-Guess Wind Speed Errors 

by P. A. Jacobs and D. P. Gaver 



Abstract 

Numerical meteorological models are used to assist in the prediction of 
weather. Each run of a numerical model produces forecasts of 
meteorological variables which are used as preliminary predictions of 
the future values of these variables. These initial predictions are 
referred to as first-guess values. Estimation of the mean-square first- 
guess error is required in the optimal interpolation process in the 
numerical prediction of atmospheric variables. Several predictors for 
the mean-square error of the first-guess wind speeds are studied. The 
results suggest that prediction using observed covariates tend to be 
better than those using first-guess covariates. However, observed 
covariates are not always available. Predictions using first-guess 
covariates are better at the 250 mb level than the 850 or 500 mb levels. 
Of those first-guess covariates studied, first-guess wind speed appears to 
be the best. 



1. INTRODUCTION AND SUMMARY 

Numerical meteorological models are used to assist in the prediction of 
weather. Each run of a numerical model produces forecasts of meteorological 
variables which are used as preliminary predictions of future values of these 
variables. These initial predictions are referred to as first-guess values. In this 
paper first-guess values will refer to the most recent 12-hour forecasts. 

In certain areas of the world, observations of forecasted variables become 
available. Prior to the next run of the numerical model a multivariate 
optimal interpolation analysis updates a first-guess value of a variable by 
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adding to it a weighted observed value of the variable if it is available. The 
weight multiplying the observed value depends on estimates of the mean- 
squared error of the first-guess value and the mean-squared error of the 
observation; cf. Goerss et al., [1991, a, b]. Thus it is of importance to predict the 
first-guess mean-squared errors. 

The general problem of modeling and predicting mean-square errors is 
important but not widely studied; see Davidian and Carroll (1987), Nelder and 
Lee (1992), Aitken (1987), McCullagh and Nelder (1983). 

In Jacobs and Gaver (1991, 1992) statistical models for the error of the first 
guess are used to predict mean-square error for first-guess wind components. 
The models assume that the error of the first guess has a normal distribution 
with mean 0 and variance which is a function that is log-linear with 
covariates. Details of the model are presented in Appendix A. 

In this paper we use data from February 1991 to compare the predictive 
ability of various models. The data consist of measurements and 12 hour 
forecasts (first-guess values) of u and v wind components at the 850 mb, 500 
mb and 250 mb pressure levels from 93 stations in North America 25N-75N 
for the month of February 1991. The forecasts are produced using the 
NOGAPS Spectral Forecast Model; cf. Hogan et al., (1991). Each station has 
measurement and first-guess values for every 12 hours; there are some 
missing observations. These missing values are deleted from the data set. The 
measurement values are subtracted from the first-guess values to obtain 
observations of the error of the first-guess value. 

Let U(o;t), (respectively V(o;t)), be the observed u-wind, (respectively 
p-wind) component at time t. Let U(/;f), (respectively V(/;0), be the first-guess 
u-wind (respectively v-wind) component at time f; (respectively V(f;t)) 
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is the forecasted value of the w-wind (respectively u-wind) component made 
12 hours previously. The first-guess error for the w-wind (respectively re- 
wind) component is 

Yu(t) = U(f;t) - U(o;t ); (respectively Yy(t) = V(f;t) - V(o;t)). (1.1) 



The following covariates are considered in the log-linear model for the mean- 
square error of the first guess. 



r{o;t) = [(U(o;t)-U{o;t-l)) 2 + {V{o; t ) - V{o; t - 1)) 2 



2 

2 



( 1 . 2 ) 



w(o;t)= U(o;t) 2 + V(o;tf 



(1.3) 



(/;') = [M/; t)-U(f;t- 1)) 2 + (V(f; t)-V(f;t- 1)) 2 ] 






(I) = [(U(/;l) - U(o;t- 1)) 2 + (V(f;t) - V(q;t- l)f] 



a(o,U;t) = |li(o; t) - U{o ; t - 1)|, a(o,V; t) = |V(o; t)-V{o,t- 1)| 
a(f, U; t) = |Lf (/; t)-U(f;t- l)|, a(f, V;t) = \ V(f; t) - V(f, t - 1)| 

a (U; t ) = \U{f; t ) - U (o; t - 1)|, a ( V; t) = \v{f; t ) - V(o, t - 1)| 



m(f;t) = max(li(/;f),V(/;f)) 



(1.4) 

(1.5) 

(1.6) 

(1.7) 

( 1 . 8 ) 
(1.9) 

( 1 . 10 ) 
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The resultant wind r(o;t), (respectively r(f;t ) and r*(t)), is a measure of the 
observed (respectively forecasted), change in the wind. The variable w(o;t), 
(respectively w(f;t)), is the observed, (respectively forecasted), wind speed. 
Higher wind speeds suggest more activity in the atmosphere. The change in 
magnitudes a(o,U;t), a{f,U;t) and a*(,U;t) (respectively a(o,V;t), a(f,V;t) and 
a*(V;t) will be used to predict Yjjit), (respectively Yy(t)). 

The data are randomly divided into two sets called DA and DB. 
Maximum likelihood estimates of the parameters of the models using 
different covariates are computed using data DA (respectively DB). 
Nonparametric models based on binning are also considered. The models are 
then used to predict the mean-square first-guess errors in data set DB 
(respectively DA). Log-likelihood functions and the empirical distribution of 
the first-guess errors normalized by their predicted mean-square errors are 
used to evaluate the models' predictive ability. Details are given in Section 2. 

In general, models which use observed covariates, e.g. w(o), a(o), have 
more predictive ability than those that use first-guess covariates, e.g. ivif), a(f), 
m{f). The models applied at the 250 mb level appear to have more predictive 
ability than those for 500 mb and 850 mb. 

Among the one-variate models for the 250 mb pressure height, the 
models that statistically appear to have the most predictive ability have as 
their covariate zv(o), a(o) or r(o). Those that have less but some predictive 
ability have as their covariate a \ zv(f), tn(f) or r\ Finally, one-variate models 
using variates r(f) and a(f) appear to have little predictive ability. Among 
those models for the 250 mb pressure height that use one first-guess covariate, 
m(/) or w(f) appear to have the most predictive ability. 
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2. THE DATA ANALYSIS 

In this section we describe the data analysis. Let U,(o;f) and If ,(/;{), 
(respectively V,(o;f) and V { be the observed and first-guess w-wind 
(respectively i?-wind) component at location i = 1, ..., S at time f. By data we 
mean the vector (Uj(o,t), Ui(f,t), Vi(o,t), Vi(f,t), D, (0,1-1), U, •(/,*-' 1), V;(o,f-l), 
Vi(f,t- 1)). The data set contains missing values. Vectors containing these 
missing values are deleted from the data set. Once missing values are deleted, 
there are 3618 vectors at the 850 mb level, 4100 at the 500 mb level, and 3744 at 
the 250 mb level. The observed values are subtracted from the first-guess 
values to obtain observations of the first-guess errors for each wind 
component 

Y i (U;t) = U,(f;t)-U i (o;t) 

Y i (V;t) = V i (f-,t)-V i (o-,t). 

The remaining data are randomly divided into two sets called DA and DB 
without regard to the values of the data, the time t, or the location. Thus, data 
from the same location for different times may be in different data sets. 
Models are estimated for each pressure level using only covariates for that 
pressure level. The covariates considered for each wind component appear in 
Appendix B. The general statistical model is described in Appendix A. 

The model is estimated using data sets DA, DB, and all the data for each 
pressure level. The estimated values for the parameters for selected models 
appear in Tables 3A, 4A, 3B, 4B, 3C and 4C. Note that the parameter estimates 
are usually positive. Hence increased values of the covariates are associated 
with higher variance of the first-guess errors. 

The models estimated from DA (respectively DB) are used to predict the 
first-guess errors in data set DB (respectively DA). One measure used for 
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assessing a model's goodness of fit and predictive ability is the value of £, the 
log-likelihood function up to addition of constants given in Appendix A 
(A. 4); the log-likelihood for predicting mean-square errors in DB using a 
model estimated using DA uses the first-guess error and covariate(s) from DB 
and the parameter estimates from DA. Values of £ are computed for data DA 
(respectively DB) using the parameters estimated using DB (respectively DA); 
these values assess each model's predictive ability. Values of £ are also 
computed for data DA (respectively DB) using parameters estimated using DA 
(respectively DB); these values assess each model's goodness of fit. 

Tables 1A, IB, 1C present the values of £ for one-variate models for the 
different pressure levels. Also displayed are the values of £ for a model in 
which the first-guess errors are independent normally distributed with mean 
0 and constant variance e a . 

Tables 2A, 2B, 2C present values for £ for two-variate models. 

Compare the value of ~£ c for the model with constant variance (no 
covariates) for DA (respectively DB) fit using DA (respectively DB) with the 
values of £ for DA (respectively DB) using models with parameters estimated 
using the other half of the data DB (respectively DA). A value of £ greater 
than ~£ c indicates that the corresponding model fit with the other half of the 
data describes the data better than the best constant variance model fit with 
the same data it is used to summarize. For 850 mb data those one-variate 
models for which £>£ c for DA and DB for both wind components are those 
with variate r\ a *, r, and xv(o). For 500 mb the one-variate models are those 
with variate r* , a(o), r, w(o), and w(f). For 250 mb, the models are those with 
variate r*, a(o), r, w(o), zv(f), and m(f). 
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To compare the predictive ability of the models, the fraction of increase in 
~i, (i-t c )/l< c \ is computed where 2 C is the maximum value of £ for the 
constant variance model (with no covariates) estimated using data DA 
(respectively DB) compared to the value of £ for DA using one-variate 
models estimated using the other half of the data DB (respectively DA). The 
values of percentage of increase appear in Table 5 for the one-variable models 
with variate r* , a, r, V)(o), wif), and m(f). Note that the fraction increase tends 
to be larger for the 250 mb for the covariates using observed data, r, w{o), and 
a(o). The fraction also tends to be larger for the first-guess covariates w(f) and 
m(f) at the 250 mb level. 

Another measure of predictability is the distribution of the first-guess 
errors divided by their predicted standard deviations. Table 6 displays the 
moments of the first-guess errors of the wind components DA (respectively 
DB) divided by the standard deviations that are predicted for it using the 
model fit using data of DB (respectively DA). Recall that the models assume 
that these errors are normally distributed with mean 0. Thus, if a model were 
perfect then the mean (respectively standard deviation, skewness, and 
kurtosis) of the normalized first-guess errors would be 0, (respectively 1, 0 and 
3). Of particular interest is the kurtosis. In this application, the kurtosis can be 
thought of as a measure of the variability of the variance (cf. Cramer page 
356). Hence, the smaller the kurtosis, the better the prediction of the model. 

Table 6 presents not only results for the model of Appendix A with 
various covariates but also results for a nonparametric one-variate model. 
This nonparametric model is as follows. The data in DA (respectively DB) are 
binned into N bins according to the value of the ordered covariate. For each 
bin, the mean of the square of the wind speed errors corresponding to the 
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covariates in that bin is computed. To evaluate the predictive ability of the 
model, the other data set DB (respectively DA) is used. The predicted mean- 
square error for a data point in DB (respectively DA) is the mean of the square 
of the wind speed errors for the bin determined from DA (respectively DB) 
the data point's covariate lies in. 

Table 6 presents selected results using data for the 250 mb level. Results 
for parametric models of Appendix A with parameters estimated by 
maximum likelihood (MLE) and the nonparametric models with bins are 
presented. Also displayed are the sample moments of the first-guess errors in 
the row labeled "none". Displayed in the row labeled "constant" are the 
sample moments for the first-guess errors divided by the predicted standard 
deviation for a model with constant variance e a fit using the other half of the 
data. 

The values of the kurtosis suggest the following. Once again, models 
using the observed covariates a(o) and r(o ) appear to make the best predictors. 
Among the first-guess covariates, m(f) and w{f) appear to have comparable 
predictive ability. If a nonparametric model using a first-guess variate is being 
considered, then using first-guess wind speed as the covariate seems to be a 
good choice. 
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APPENDIX A 



THE STATISTICAL MODEL 



In this Appendix we describe the statistical model. Let Uj(o;t) (respectively 
Vj(f;t)) denote the observed u-wind component (respectively first-guess wind 
component) at location i at time t; i = 1, S. Let Vi(o;t), (respectively V,(/;f)) 
denote the observed u-wind component (respectively first-guess wind 
component) at location i at time t. The first-guess error of the w-wind 
(respectively u-wind) component at location i at time t is 



(respectively, (A.l) 

Y i (V-,t) = V i (o;t)-V i (f;t)). 



The model is that { Yi(U;t ), i = S) and {Y,-(V;f), i = 1, S) are 

independent random variables having a normal distribution with mean 0. 
The variance of Y,(L/;f) is log-linear with a number of co variates. That is 






= exp 



a + SW *«(/;0 



(A.2) 



The likelihood function for this model is (up to multiplication by 
constants) 



P p ) 



=nih*P 

t i 



a + X£/(0*i(;;0 



-I'l 




' / 




1 2 




>exp< 


--y exp- 








. v 



M 

The log-likelihood function is (up to addition by constants) 



(A.3) 
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}(a f pi,...,p p ) 

= IS” 



t l 



«+ X /*/( 0*»(/ ; 0 

M 





' 


1 2 




--y exp< 




J 


. V 



W 



a +'ZPj(t)xi(j;t) 



(A. 4) 



The recursive procedure used to estimate the parameters (a, fi\, ftp) is 
described in Gaver and Jacobs [1991]. 
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APPENDIX B 



THE CO VARIATES 



In this Appendix we list the covariates that were considered. As before let 
Uj(o;t ) and Uj(f;t), (respectively V,(o;f) and V{(f;t )) denote the observed and 
first-guess w-wind (respectively p-wind) component at time t for location i, 
i = 1 , S. The covariates considered for the first-guess error of the u-wind 
component are 



« i (o,U;/) = |U((o;/)-i7 1 -(o;f-l)| 
(U; () = |Lf, (/; f) - U, (o;i - 1)[ 

io,(o;l) = [Ui(o;() 2 +V , i(o;<) 2 ] 



»/(/;<)= +Vi(f; 




n(o;t) = 



2 

[U;M- - 1)] 2 +[Vi(o;()- - 1)] 2 ] 2 



= [[U((/; 0 - U,(/;l - 1 )] 2 + [v,(o;0- - l)f ] 2 

2 

'i'(') = [[u,- (/;<)- +[vi(/;0- H(»;' - i)f ] 2 

m(/; t) = max(H (/; f),V(/; t )). 



The covariates considered for the first-guess error of the p-wind 
component are 
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a i(o,V;t) = \Vi(o;t)~ Vj(o;t- 1)| 






w,(o;t),w,(f;t),r i (o;t),r i (f;t), and r'(t) 
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r{t)=[(U(o;t)-U(o;t- 1)) 2 +(V(o;t)~ V{o; t- 1)) 2 
w(o;t) =\u(o;t) 2 +V(o;t) 2 ] ' 



TABLE 3 A 
One- Variate Models 
Parameter Estimates 
(Standard Errors) 
850 mb. February Data 
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' M = [(U(/; I) - U(o; I - if + (V(f; l) - V(o; t - 1)) 2 ] 
a(o;I) = |U(o;l)-U(o;i- 1)|, «*(() = \U{f;t)-U(o;t- 1)| 



Parameter Estimates 
(Standard Errors) 
850 mb. February Data 
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r* (() = [(U(/; () - U(o; t - 1)) 2 + (V(f; l) - V(o; 1 - l)f ] 
a(o;t) = |U(o ;l)-U(o;l- 1)|, a'(t) = p(f;l)-U(o;l- 1)| 
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r(t) = [(U(o; t)-U(o;t- 1)) 2 + ( V(o; t) - V{o; t - 1)) 
w{o;t) =\u{o;t) 2 + V{o;t) 2 ] W2 



TABLE 3B 
One-Variate Models 
Parameter Estimates 
(Standard Errors) 
500 mb. February Data 
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r *(0 = [(U(/;l)- U(o;t - 1)) 2 + (V(f;t)-V(o;t - 1)) 2 ] 
i(o;() = |U(o;f)-U(o;f-l)|, a'(t) = |U(/;f)-U(o;I- 1)| 



TABLE 4B 
Two-Variate Models 
Parameter Estimates 
(Standard Errors) 
500 mb. February Data 
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r (0 = |M/; t) - U(o; t - 1)) + (V(f;t)~ V(o;t- l)f J 
(o;t) = \U(o;t)-U(o;t- 1)|, a (t) = \ll{f;t)-U(o;t-l)\ 
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r{t) = [(li(o; t)-U(o;t- 1)) 2 + (V(o; t) - V{o; t - 1)) ! 
w(o;t)=[u(o;t) 2 +V{o;t) 2 ^ /2 

w(/;0 =[u{f;t) 2 + v (f;t) 2 ] ' 

m(f;t) =ma x(U(f;t),V(f;t)) 
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r(l) = [(U(o;f) - U(o;t - l)) 2 + (V(<r.l) - V[o; 1 - l)) 2 
iv{o;t) =\u(o-,tf + V(o;l) 2 1' /2 



TABLE 3C 
One-Variate Models 
Parameter Estimates 
(Standard Errors) 
250 mb. February Data 
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TABLE 4C 
Two-Variate Models 
Parameter Estimates 
(Standard Errors) 
250 mb. February Data 
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One-Variate Models 
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TABLE 6 

Sample Moments of 

First-Guess Wind Speed Errors Divided by Predicted Standard Deviations 
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<{f;t) = max(U(f;t),V(f;t)) 

</;') =[u(/;0 2 +v(/;') 2 ]' 



INITIAL DISTRIBUTION LIST 



1. Research Office (Code 08) 1 

Naval Postgraduate School 

Monterey, CA 93943-5000 

2. Dudley Knox Library (Code 52) 2 

Naval Postgraduate School 

Monterey, CA 93943-5002 

3. Defense Technical Information Center 2 

Cameron Station 

Alexandria, VA 22314 

4. Department of Operations Research (Code OR) 1 

Naval Postgraduate School 

Monterey, CA 93943-5000 

5. Prof. Donald Gaver (Code OR/Gv) 5 

Naval Postgraduate School 

Monterey, CA 93943-5000 

6. Prof. Patricia Jacobs (Code OR/Jc) 5 

Naval Postgraduate School 

Monterey, CA 93943-5000 

7. Center for Naval Analyses 1 

4401 Ford Avenue 

Alexandria, VA 22302-0268 

8. Dr. J. Abrahams, Code 1111, Room 607 1 

Mathematical Sciences Division, Office of Naval Research 

800 North Quincy Street 
Arlington, VA 22217-5000 

9. Dr. David Brillinger 1 

Statistics Department 

University of California 
Berkeley, CA 94720 



28 



10. Prof. Brad Carlin 1 

School of Public Health 

University of Minnesota 
Mayo Bldg. A460 
Minneapolis, MN 55455 

11. Prof. H. Chemoff 1 

Department of Statistics 

Harvard University 
1 Oxford Street 
Cambridge, MA 02138 

12. Dr. John Copas 1 

Dept, of Mathematics, 

University of Birmingham 
P. O. Box 363 
Birmingham B15 2TT 
ENGLAND 

13. Professor Sir David Cox 1 

Nuffield College 

Oxford, OXI INF 
ENGLAND 

14. Professor H. G. Daellenbach 1 

Department of Operations Research 

University of Canterbury 
Christchurch, NEW ZEALAND 

15. Dr. S. R. Dalai 1 

Bellcore 

445 South Street 
Morristown, NJ 07962-1910 

16. Dr. D. F. Daley 1 

Statistic Dept. (I.A.S.) 

Australian National University 
Canberra, A.C.T. 2606 
AUSTRALIA 



29 



17. Prof. Bradley Efron 1 

Statistics Dept. 

Sequoia Hall 
Stanford University 
Stanford, CA 94305 

18. Prof. George S. Fishman 1 

Curr. in OR & Systems Analysis 

University of North Carolina 
Chapel Hill, NC 20742 

19. Dr. Neil Gerr 1 

Office of Naval Research 

Arlington, VA 22217 

20. Dr. J. Goerss 10 

Naval Oceanographic and Atmospheric Laboratory 

Monterey, CA 93943-5006 

21. Dr. D. C. Hoaglin 1 

Department of Statistics 

Harvard University 
1 Oxford Street 
Cambridge, MA 02138 

22. Institute for Defense Analysis 1 

1800 North Beauregard 

Alexandria, VA 22311 

23. Prof. J. B. Kadane 1 

Dept, of Statistics 

Carnegie-Mellon University 
Pittsburgh, PA 15213 

24. Dr. Jon Kettenring 1 

Bellcore 

445 South Street 

Morris Township, NJ 07962-1910 

25. Koh Peng Kong 1 

OA Branch, DSO 

Ministry of Defense 
Blk 29 Middlesex Road 
SINGAPORE 1024 



30 



1 



26. Dr. A. J. Lawrence 

Dept, of Mathematics, 

University of Birmingham 
P. O. Box 363 
Birmingham B15 2TT 
ENGLAND 

27. Prof. M. Leadbetter 1 

Department of Statistics 

University of North Carolina 
Chapel Hill, NC 27514 

28. Prof. J. Lehoczky 1 

Department of Statistics 

Carnegie-Mellon University 
Pittsburgh, PA 15213 

29. Dr. Colin Mallows 1 

AT&T Bell Telephone Laboratories 

600 Mountain Avenue 
Murray Hill, NJ 07974 

30. Prof. M. Mazumdar 1 

Dept, of Industrial Engineering 

University of Pittsburgh 
Pittsburgh, PA 15235 

31. Prof. Carl N. Morris 1 

Statistics Department 

Harvard University 
1 Oxford St. 

Cambridge, MA 02138 

32. Prof. F. W. Mosteller 1 

Department of Statistics 

Harvard University 
1 Oxford St. 

Cambridge, MA 02138 

33. Operations Research Center, Rm E40-164 1 

Massachusetts Institute of Technology 

Attn: R. C. Larson and J. F. Shapiro 
Cambridge, MA 02139 



31 



34. Prof. Frank Samaniego 1 

Statistics Department 

University of California 
Davis, CA 95616 

35. Prof. W. R. Schucany 1 

Dept, of Statistics 

Southern Methodist University 
Dallas, TX 75222 

36. Prof. H. Solomon 1 



Department of Statistics 
Sequoia Hall 
Stanford University 
Stanford, CA 94305 



37. Prof. W. Stuetzle 1 

Department of Statistics 

University of Washington 
Seattle, WA 98195 

38. Prof. J. R. Thompson 1 

Dept, of Mathematical Science 

Rice University 
Houston, TX 77001 

39. Prof. J. W. Tukey 1 

Statistics Dept., Fine Hall 

Princeton University 
Princeton, NJ 08540 



40. Dr. D. Vere-Jones 1 

Dept, of Math, Victoria Univ. of Wellington 

P. O. Box 196 
Wellington 
NEW ZEALAND 

41. Prof. David L. Wallace 1 

Statistics Dept., University of Chicago 

5734 S. University Ave. 

Chicago, IL 60637 



32 



42. Dr. Ed Wegman 1 

George Mason University 

Fairfax, VA 22030 

43. Dr. P. Welch 1 

IBM Research Laboratory 

Yorktown Heights, NY 10598 

44. Prof. Roy Welsch 1 

Sloan School 

M.I.T. 

Cambridge, MA 02139 



33 



DUDLEY KNOX LIBRARY 




3 2768 003371 88 1 



